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ON THE CONTROLLED EIGENVALUE PROBLEM FOR STOCHASTICALLY 
PERTURBED MULTI-CHANNEL SYSTEMS * 

GETACHEW K. BEFEKADUt 


Abstract. In this brief paper, we consider the problem of minimizing the asymptotic exit rate of diffusion 
processes from an open connected bounded set pertaining to a multi-channel system with small random perturbations. 
Specifically, we establish a connection between: (i) the existence of an invariant set for the unperturbed multi-channel 
system w.r.t. certain class of state-feedback controllers; and (ii) the asymptotic behavior of the principal eigenvalues 
and the solutions of the Hamilton-Jacobi-Bellman (HJB) equations corresponding to a family of singularly perturbed 
elliptic operators. Finally, we provide a sufficient condition for the existence of a Pareto equilibrium (i.e., a set of 
optimal exit rates w.r.t. each of input channels) for the HJB equations - where the latter correspond to a family of 
nonlinear controlled eigenvalue problems. 
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1. Introduction. Consider the following continuous-time multi-channel systerrQ 

E TI 

x{0) = xo, ( 1 . 1 ) 

i—1 


where a; € A C is the state of the system, Ui €h(i C R'"* is the control input to the ith- 
channel, and A G R^^'^ and Bi G for i = 1, 2,... , n, are constant matrices. 

Let £) C A be an open connected bounded set with smooth boundary. For the multi-channel 
system in (11.11) . we consider the following class of state-feedback controllers 


r C 


{( 


71 , 72 , 


,7n) e n 


n 




( 1 . 2 ) 


where A{D) C D U dD is the maximal invariant set (under the action of S'* = exp{ (A -b 
YJi=i f > 0) such that 

= K{D), Vf>o, (1.3) 


for any set Q,. 


In what follows, we provide a connection between the existence of an invariant set for the 
system S* in Z? U dD and the asymptotic behavior of the principal eigenvalue for singu¬ 
larly perturbed elliptic operator which is associated with the following stochastic differential 
equation (SDE) 




where 
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- is an R'^-valued diffusion process, e is a small parameter lying in an inter¬ 
val (0,e*) (which represents the level of random perturbation in the system), and 
(71,72,..., 7„) is an n-tuple of state-feedbacks from the class F, 

- (t: —>■ is Lipschitz continuous with the least eigenvalue of cr(-)cr^(-) uni¬ 

formly bounded away from zero, i.e., 

a{x)a'^{x) > Kidxd, Vx e W^, 


for some k > 0, and 

- W{-) (with W (0) = 0) is an d-dimensional standard Wiener process. 

For any fixed e € (0, e*), let be the first exit time of the diffusion process from 

the set D, i.e.. 


= infjf > 0 



(1.5) 


Further, let \^A} and usual, denote the probability of an even A and the 

expectation of a random variable respectively, for the diffusion process starting 

from xq G D. 

Here, it is worth mentioning that, in system reliability analysis and other studies, one often 
requires to confine a controlled diffusion process to a given open connected bounded 

set D as long as possible. A standard formulation for such a problem is to maximize the mean 
exit time from the set D. We also observe that a more suitable objective would 

be to minimize the asymptotic rate with which the diffusion process X^’^{t) exits from the 
set D. Further, this suggests minimizing the principal eigenvalue 


0 = - 1™ sup i log I> 11, 

t^OO t 'I J 


( 1 . 6 ) 


with respect to certain class of admissible controls (including the above class of state-feedbacks 
in (ll.2b )n Note that if the domain D contains an equilibrium point for the deterministic multi¬ 
channel system in ( 11.11 ). when such a system is composed with the n-tuple of state-feedbacks 
from r, then the principal eigenvalue g, which is associated with the singularly perturbed 
elliptic operator 


-'^£.o(-)(^) = + -f |tr|cr(x)cr'^(x)v2(.)|, 

with zero boundary conditions on dD, satisfies (see also Corollary 12.3b 


— logAgQ = e ^r'^ + o(e as 


0 , 


where r~^ is given by the following 


= limsup inf ^{t)GDLIdD, <G[0,r] 

T^oo vWei' J 


(1.7) 


( 1 . 8 ) 


^Recently, the authors in (T] have provided interesting results on controlled equilibrium selection in stochasti- 
cally perturbed dynamics, but in a slightly different context. 





On the controlled eigenvalue problem 
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and the action functional with (71,72, ■ ■ •, 7 n) G T, is given by 




d(p{t) 


1 T 


-1 


dt 


dt, (1.9) 




where the set consisting of all absolutely continuous functions (p € C't([0, T], R'^), with 
(/?(0) = xq, a compact set in D (e.g., see ifT^ . ifTSl . ||8l Chapter 14] or Q for additional 
discussions). 

On the other hand, if the maximum invariant set A(i9) for the deterministic multi-channel 
system in (II.lb . w.r.t. the state-feedbacks from T, is nonempty. Then, the following asymp¬ 
totic condition also holds true (see Lemma 12.51 in the Appendix section (cf. mni Theo¬ 
rem 2.1])) 

— lim limsup - logP^^n > f| < 00 , xo £ D. (LIO) 

£->•0 t_>oo t ’ ^ 


Moreover, the principal eigenvalue XI g turns out to be the boundary value between those R < 
r~< for which \^exp{e~^< 00 and those R > r"^ for which £’^g’^{exp(e“^ 
PP’T')} = 00 (see also lH] pp. 373-382] for additional discussions). Note that estimating 
the asymptotic exit rate with which the controlled-diffusion process exits from the 

domain D is also related to a singularly perturbed eigenvalue problem. For example, the 
asymptotic behavior for the principal eigenvalue corresponding to the following eigenvalue 
problem 

(x) + i^) = vx e p 

■0*. (x) =0, Vx £ dD 

where 0*. £ {D)r\C(D\JdD), forp > 2, with 0*. > 0 on 79, has been well studied in 
the past (e.g., see 0 in the context of an asymptotic behavior for the principal eigenfunction; 
and see 0 or 0 in the context of an asymptotic behavior for the equilibrium density). 

Before concluding this section, let us introduce the following definition for minimal action 
state-feedbacks, w.r.t. the action functional 10 (p), which is useful for the development of our 
main result. 

Definition 1.1. The n-tuple (71,721 ;70) £ T B said to be minimal action state- 

feedbacks if 

e argmin/0(p). (1.12) 


( 1 . 11 ) 


In the following section, we present our main results - where we establish a connection be¬ 
tween the asymptotic exit rate with which the controlled diffusion process (w.r.t. each of the 
input channels) exits from the set D and the asymptotic behavior of principal eigenvalues for a 
family of singularly perturbed elliptic operators with zero boundary condition on dD. Later, 
such a formulation allows us to provide a sufficient condition for the existence of a Pareto 
equilibrium (i.e., a set of optimal exit rates w.r.t. each of the input channels) for the HJB equa¬ 
tions - where the latter correspond to a family of nonlinear controlled eigenvalue problems 
(e.g., see or 191 Chapter 8] for additional discussions on eigenvalue problems). 
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2. Main Results. In this section, we consider a family of SDKs (w.r.t. each of input 
channels) 


= (^ + V . . + B,u^{t)dt + yfea{X^'\t))dW{t), 

X^’*(0) = a:o, i=l,2,...,n, (2.1) 


where Ui{-) is a Z/f^-valued measurable control process to the ith-channel (i.e., an admissible 
control from the set Ui C K.’’0 such that for all t > s,W{t) — W(s) is independent of Ui(iy) 
for ly < s (nonanticipativity condition) and 


E / |ui(r)pdr < oo, 


Vt > s, 


for i = 1,2,... ,n. 
Next, let 


\U _ f\Ul \U2 \Un \ 


with 


limsupilogP,^“’“‘{T; >t}, i = l,2,...,n, 

t^OO C 


( 2 . 2 ) 


where the probability is conditioned on the initial point xo G D and the admissible 

control Ui GlAi. Moreover, r/ is the first exit time for the diffusion process from the 

set D, i.e., r/ = inf {f > 0 | ^ P} for i = 1, 2,..., n. 

Further, let us introduce the following set 





(71 > 72 , 



(2.3) 


Remark 2.1. Notice that the set (w.r.f. the class of state-feedbacks T) is a closed subset 
o/R". 

Define the partial ordering ^ on by A“ ^ A“ , i.e.. 


/.Uj ^ \“2 \“n'l 

2\g_2> ■ ■ • > 2'e,n) ^ (^£,1, ^^£,2, ' • ' > 


(2.4) 


/ // 

if A“‘j < A“*j for alH = 1, 2,..., n, with strict inequality for at least one i € {1,2,..., n}. 
Further, we say that A“ € is a Pareto equilibrium (w.r.t. the class of state-feedbacks F) if 
there is no A“ G E*^ for which A“ ^ A“*. 

Then, we have the following proposition that provides a sufficient condition for the exis¬ 
tence of a Pareto equilibrium A“ G E*^ (i.e., a set of optimal exit rates w.r.t. each of input 
channels). 

Proposition 2.2. Suppose that the statement in Definition \l.l\ holds true at least for one 
n-tuple of state-feedbacks (7i, 72, ■ ■ ■ > 7n) G T. Then, there exists a Pareto equilibrium 
A“ G E*^ (i.e., a set of optimal exit rates w.r.t. each of input channels) such that 


(A“\, A“i ..., A< ) ^ (A“\, A“= 2 , ..., ) on E^ 


(2.5) 



On the controlled eigenvalue problem 
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where the principal eigenvalues are the unique solutions for the HJB equations corre¬ 
sponding to the following family of nonlinear controlled eigenvalue problems 

max„,gii, tjj*. (x, Ui) + A“^ V'u. i^) = o|, Vx G D ^2 6) 

'fu-ix) =0, Vx G dD 

with G VFjoc (-D) n C{D U dD),forp > 2, with . > 0 on D, and 

/ori= 1,2, 

Proof. Suppose there exists an n-tuple of state-feedbacks ( 71 , 72 , ■ ■ ■ ,Jn) £ T that satisfies 
the statement in Definition ll.il Then, our first claim for G (ll) fl (7(11 U dD^, with 

p > 2, follows from Equation (12.6b (cf. ifTTl Theorems l.i, 1.2 and 1.4]). That is, if u* for 
i G {1,2,..., N}, is a measurable selector of argmax{£“l^/>*^ (x)}, with x £ D. Then, by 
the uniqueness claim for the eigenvalue problem (cf. Equation (11.1 lb ), we have 

= - 1 ™ sup i log |r{ > f I, (2.8) 


where the probability is conditioned on Xq, u * and (j*,... ... ,7]^). 

Moreover, for any other admissible controls Vi &Ui, we have 


W < o> vf > 0. 


(2.9) 


Let (5 C be a smooth bounded open domain containing D U dD. Let ipy. and be the 
principal eigenfunction-eigenvalue pairs for the eigenvalue problem of on dQ. Further, 
let 


= inf|f > 0 I ^ (jj. 

Then, under Vi, for i G (1,2,..., N} and for some x G P, we have 

ifvi {x) > I exp(A^lf) (x(f)) l|ff > 1, 


> inf 

x£D 


Leading to 


> - lim sup i log {rt>t}. 

t^OO t ' ^ j 


( 2 . 10 ) 


i’vi{x{t)) exp(A^i-f)P^'(-’“'> f|. (2.11) 


( 2 . 12 ) 


Letting Q shrink to D and using Proposition 4.10 of CD, then we have A^f —.Thus, we 
have 


A“U - lim sup i log {rf > f}. 

i-b-(50 c ^ J 


(2.13) 


Combining with Equation (12.8b . this establishes the optimality of u* and the fact that is 
the optimal exit rate. 
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Conversely, let v* be an admissible optimal control, then we have 

(2.14) 

and 

^elru^i^,v:)+x:iru^{x)<o, yt>o, has) 

with = A“‘j, for i = 1, 2,..., n. 

Further, notice that is a scalar multiple of -0*. and, at some x G D (cf. IfTTl The¬ 
orem 1.4(a)]). Then, we see that v* is also a maximizing measurable selector in Equa¬ 
tion ( 12.6b . 

On the other hand, for a fixed small e, define the following continuous functional (i.e., a utility 
function over a closed set G M" that can be convexfied) 

9 A“ C/'=(A“) = (w, A“) e R, (2.16) 


where Wi > 0 for i = 1,2,... ,n. 

Note that the utility function satisfies the property (A“ ) < C/'^(A“”), whenever A“' ^ 
A“ on E*^ w.r.t. the class of state-feedbacks F. Then, from the Arrow-Barankin-Blackwell 
theorem (e.g., see ||2l), one can see that the set in 


A“ e E'^ 


3a;i > 0, i = 1,2,... ,n, 


min (u), A“) = (uj. 



(2.17) 


is dense in the set of all Pareto equilibria. This further implies that, for any choice of uJi > 0, 
i = 1, 2,..., n, the minimizer (w, A“) = satisfies the Pareto equi¬ 

librium condition w.r.t. some minimal action state-feedbacks ( 71 , 72 , ■ ■ ■ ,ln) ^ T. This 
completes the proof of Proposition [ZS] □ 

We conclude this section with the following corollary that gives a lower-bound for in 
dl.Sb (w.r.t. some minimal action state-feedbacks ( 7 *, 72 , • • ■, 7j^) £ T) (cf. (S] Corol¬ 
lary 11.2,pp. 377]) 

Corollary 2.3. Suppose that the statement in ProDosition \2.2\ holds. Then, r'^ in ( Ob , 
w.r.t. the minimal action state-feedbacks ( 7 *, 72 , ■ ■ ■, 7n) £ T, satisfied 

> max r“S (2.18) 


'^Notice that 


where 


It 


P = limsup inf 

T^-oo T 


{vlf) = ^ ^ ~ (^ + XlLl Mi))] 




e D U dD, t G [0 


.r]!, 




dt. 










On the controlled eigenvalue problem 
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where is given by 


= — elogA“*j as e —>■ 0. 


(2.19) 


Remark 2.4. Here, we remark that, for each i G {1,2,..., n}, we have 



which suggests that the above corollary is useful for selecting the most appropriate minimal 
action state-feedbacks from F that confine the diffusion process to the prescribed set 

D for a longer duration (cf Equations Oi l and (lO l). 

Appendix. The following lemma (whose proof is an adaptation of ifTOl Theorem 2.1]) 
provides a condition under which the maximal invariant set K{D) is nonempty. 

Lemma 2.5. 

(a) If for some xq € D, 

— limsuplimsup - logP^Q’^jr'^’^ > < oo, (A.l) 

e-rO t-t-oo t ’1 1 

then the maximal invariant set A(Z1) is nonempty. 

(b) If for some xq G D, 


limsuplogi?fo’^<r'^’'’'!• = oo. 


(A.2) 


then the maximal invariant set A(Z1) is nonempty. 

Proof. Suppose that the maximum closed invariant set A(D) is empty (i.e., the invariant set 
for S'* in D U dD is empty). Then, there exists an open bounded domain D Z) DU dD such 
that the corresponding set A{D) is also empty. 

Note that it is easy to check that if D 2 C Di, then Ai^Df) C A{Di). Take the following 
sequence { } of open domains such that 




(A.3) 


If A{Dm) f 0 for all m > 1, then 



(A.4) 


Moreover, since A{Dm) is closed, we have 


A(Di) D A{D2) D A(D3) d ••• . 


(A.5) 


Note that A is an invariant closed set for the unperturbed system 
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and A D Du dD. Thus, 0 7 ^ A C A{D). This contradicts our earlier assumption. Then, for 
some mo > 1 , we have 

AiDmo) = 0. (A. 6 ) 


Let D = D^g, for any e £ (0, e*) and Xq G DU dD, introduce the following 

= inf> 0 I exp| (a + ^ D U Dd'^. (A.7) 

Then, we can show that < 00 for any xq G D U dD. 

Notice that, if = 00 , then D U dD ^ exp| (^A + for t>0. Then, 

we have the following 

I exp ^ f > o| C A{b) = 0 , (A. 8 ) 

which show that bD is hnite. 

Note that, from upper-semicontinuity of we have 

T = sup < 00 . (A.9) 

XQ^DudD 

Moreover, for any i5 > 0, let 

liin sup i7^o’'’'|dist(x*^’'’'(f), exp| (a + ^ ~ ^ — (A-IO) 

From Equations ( lA.Tb - dA.lOl ). we have 

sup > r| ^ 0 as e ^ 0. (A.ll) 

XqGD ’ ^ ^ 


Then, using the Markov property, we have 

71)2:0,7/_e,7 \ 77^0’'^ 

7,0 J — A^ o Xt'.T'>tAj 0 Xt^’^>T ■ ■ ■ Be,0 Xt'.'t>T, 


< ( sup > t}) , 

^xoGD *■ r / 


(A. 12) 


where XA is the indicator for the event A. 

Since > f| decreases in t, then we have 

limsupilogP;g’^|7'^ > t| < ;^log( sup > t}). (A.13) 

t->oo c ’ t J X ^XqGD ’ t J / 

Taking into account Equation (lA.llI) . then, for any xg G D, we have the following 

limsup i logP^7’"’^{7’^ > —>■ —oo as e —>■ 0. (A. 14) 

t-)-oo t ’ L J 


Hence, our assumption that A{D) = 0 is inconsistent. 






On the controlled eigenvalue problem 
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To proof part (ii), notice that 


^ >{£- 1)t}. (A. 15) 

Assumption A{D) = 0 gives, in view of Equations ( lA.l 11 1. ( IA.12I I and ( IA.13I ) for sufficiently 
small e > 0 and for any xq € D, 


E- 




=1 


sup 

^xqGD 


JD^O 

^e,0 


> t} 


1 ^-1 


< oo, 


(A. 16) 


which contradicts with Equation (IA.2l i. This completes the proof of Lemma 12. 5 1 □ 
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